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CHAPTER  I 

INTRODUCTION 

Statement  of  the  Problem 

Every  year  the  U.S.  Navy  spends  thousands  of  dollars 
on  training  programs  to  increase  the  effectiveness  and 
leadership  abilities  of  the  supervisors  within  the  Naval 
Civilian  Personnel  Command.  Presently  there  is  no  means 
to  measure  the  degree  to  which  these  courses  are  meeting 
their  instructional  objectives.  In  most  cases-  the  sole 
qualitative  and  quantitative  evaluation  of  the  courses 
comes  from  end-of-course  critiques  completed  by  the 
attendees  . 


Purpose  of  the  Stud-' 


The  purpose  of  this  study  was  to  take  one  of  these 
courses  and  attempt  to  measure  the  degree  to  which  it 
was  accomplishing  its  objectives.  Specifically,  the 
Basic  Supervisory  Development  Seminar  was  evaluated  under 
a  two  phase  project.  Phase  I  involved  a  survey  of  past 
attendees  to  determine  if  adequate  emphasis  was  placed  on 
the  course  content  to  meet  their  needs  as  supervisors 
cnee  back  on  the  job.  Phase  II  involved  an  evaluation  of 


'three  different  sessions  of  the  seminar  to  determine  if 
it  had  a  significant  influence  on  the  leadership  styles 
of  its  attendees. 

Assumptions 

It  was  assumed  that  all  individuals  answered  the 
various  questionnaires  candidly  and  to  the  best  of  their 
ability . 


Definition  of  Terms 

For  the  purpose  of  this  study  the  following  terms 
will  be  defined  as  follows: 

Concern  for  people  -  Refers  to  a  leader's  interest 
in  those  behaviors  that  affect  the  welfare  of  his  or 
her  subordinates. 

Concern  for  task  -  Refers  to  a  leader's  interest  in 
those  behaviors  that  initiate  structure  and  more  clearly 
define  the  task  involved. 

Leadership  style  -  The  relationship  between  a 
leader's  "concern  for  task"  and  "concern  for  people"  as 
determined  by  the  T-P  Leadership  Questionnaire  (Pfeiffer 


&  Jones,  197*0. 
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CHAPTER  II 

REVIEW  OF  THE  LITERATURE 
Historical  Perspective 

Before  any  attempt  can  be  made  to  evaluate  the 
effectiveness  of  a  leadership  training  program,  a  basic 
understanding  of  current  leadership  theories  is  required. 
In  examining  the  wealth  of  information  on  the  subject, 
it  becomes  apparent  that  today  the  field  of  leadership 
theory  is  as  filled  with  controversy  as  it  has  been 
since  the  turn  of  the  century. 

At  the  turn  of  the  century  the  first  attempts  at 
scientif ically  defining  leadership  theory  were  being 
made  by  the  trait  theorists.  It  was  their  contention 
that  effective  leaders  were  born  with  certain  innate 
abilities  and  characteristics  that  distinguished  them 
from  their  less  effective  counterparts.  If  these 
desired  qualities  could  be  identified,  they  could  form 
the  basis  for  future  selection  of  successful  leaders 
with  far  more  certainty  than  trying  to  train  someone 
without  them.  Unfortunately,  after  almost  a  half 
century  of  intense  research,  no  scientific  consensus 
emerged  as  to  a  definitive  list  of  these  much 


sought.-al  ter  traits. 

It  was  also  at  the  turn  of  the  century  that  effici¬ 
ency  experts  like  Fredrick  Taylor  made  the  first  sys¬ 
tematic  attempts  to  study  management  and  organizations. 
The  goal  of  Taylor’s  "Scientific  Method"  was  to  reduce 
all  tasks  to  a  series  of  mechanical  movements  which  could 
be  analyzed  and  streamlined  into  the  simplest,  "test  way" 
to  do  the  job.  During  the  first  quarter  of  the  twentieth 
century,  the  popularity  of  this  philosophy  J ad  to  a 
strong  emphasis  on  performance  and  productivity.  It  was 
believed  that  the  primary  function  of  managers  was  to 
set  standards  and  see  that  employees  meet  those  standards 
(Dreilinger,  McElheny  &  Robinson,  1982). 

While  the  scientific  method  did  make  significant 
contributions  to  the  industrialization  process,  it 
failed  to  solve  the  problems  of  low  productivity  and 
worker  interest.  For  this  reason,  in  1924  the  now  famous 
Hawthorne  experiments  were  launched.  During  various 
experiments  conducted  at  the  Hawthorne  Works  of  the 
Western  Electric  Company  in  Chicago  productivity  soared. 
However,  no  correlation  could  be  determined  between  the 
productivity  and  the  variables  that  were  altered  within 
the  physical  conditions  of  the  assembly  lines.  After 
exhaustive  interviews  with  the  subjects  involved,  it  was 


discovered  that  the  incentive  and  motivation  of  the 
workers  were  the  key  to  the  increased  productivity. 

The  unaccustomed  attention  the  workers  received  while 
being  consulted  for  ideas  made  them  and  their  jobs 
seem  important  for  the  first  time.  The  significance 
of  these  results  in  the  history  of  modern  management 
cannot  be  over-exaggerated  (Owens,  1 9 8 1  ) .  After 
Hawthorne ,  every  serious  management  study  had  to  become 
two-dimensional,  taking  into  account  the  psychological 
and  human  aspects  as  well  as  the  physical  properties  of 
the  workplace  and  worker. 

As  a  result  of  the  Hawthorne  studies  a  major  shift 
occurred  in  the  direction  of  a  human  relations  orienta¬ 
tion  for  management.  Now  the  manager's  primary  function 
became  the  motivation  of  employees  in  order  to  accomplish 
the  organization's  goals  (Dreilinger,  et  al.,  1982). 

During  this  same  period,  theorists  began  to  shift 
to  analysis  of  effective  leader  "behaviors"  rather  than 
traits.  It  was  felt  that  behaviors,  unlike  traits,  were 
observable  ,  more  exactly  defined,  and  therefore,  more 
amenable  to  the  methods  of  empirical  research. 

These  behavior  theorists  were  actively  pursuing 
research  designed  to  categorize  effective  leader  behaviors 
Eventually,  all  of  these  behaviors  were  reduced  to  two 


general  categories:  "initiating  structure"  and  "con¬ 
sideration."  Structure  behaviors  involved  clearly 
stating  goals  to  achieve work  assignments,  job  struc¬ 
tures,  and  other  factors  that  let  the  workers  know 
what  was  expected  of  them.  Consideration  behaviors  were 
those  related  to  the  concern  for,  and  personal  interest 
in,  subordinates  and  their  welfare.  As  the  theorists 
categorized  these  behaviors  they  began  to  notice  that 
different  types  of  leaders  shared  certain  groups  of  these 
behaviors.  Leadership  style  became  the  term  used  to 
define  a  relatively  enduring  set  of  these  behaviors  which 
were  characteristic  of  the  individual  regardless  of  the 
situation  (Feidler,  197*0. 

By  the  1960’s  emphasis  in  the  field  had  shifted 
again,  this  time  from  behaviors  to  styles.  Several 
leadership  style  theories  developed  that  attempted  to 
integrate  the  concerns  for  structure  and  consideration. 
McGregor  postulated  that  there  were  two  general  types 
of  organization  climates  that  influence  leadership 
styles.  In  the  traditional,  "Theory  X"  organization, 
the  manager's  role  prescription  is  based  on  the  premise 
that  workers  are  basically  lazy,  irresponsible  and 
disloyal,  and  therefore  require  a  structuring,  auto¬ 
cratic  supervisor.  In  contrast,  a  "Theory  Y" 


which  McGregor  favors,  is  based  on  the  premise  that 
workers  are  inherently  motivated,  mature,  and  will  do 
a  good  job  if  provided  with  an  environment  based  on 
democratic  procedures,  participative  decision-making, 
and  self-control  (Fiedler,  1974). 

Another  theory  which  shares  the  participative 
philosophy  of  Theory  Y  is  Blake  and  Mouton 's  Managerial 
Grid .  Their  theory  is  based  on  a  two-variable  leader¬ 
ship  model  which  measures  a  leader's  orientation  towards 
people  and  task.  It  characterizes  the  "9,9"  leadership 
style,  which  maximizes  both  concerns,  as  the  single 
best  leadership  style  (Blake  &  Mouton,  1982). 

Meanwhile,  Hersey  and  Blanchard  introduced  their 
Situational  Theory  of  Management  and  started  a  con¬ 
troversy  with  the  Managerial  Grid  proponents  that 
continues  today.  Situa tionalism  is  similarly  based  on 
the  two  variables  of  task  and  relationship  orientation. 
However,  in  contrast  to  the  Managerial  Grid's  "one 
best  style,"  their  model  involves  four  leadership 
styles,  the  appropriateness  of  which  depends  on  the 
maturity  of  the  subordinates.  These  are  high  task/high 
relationship ,  high  task/low  relationship,  low  task/high 
rela tionship ,  and  low  task/low  relationship  (Blake  & 
Mouton  ,  1974). 
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The  most  recent  major  contribution  to  the  leadership 
field  is  Fiedler's  Contingency  Theory.  He  questions  the 
individual's  ability  to  change  his  or  her  leadership 
style  and  suggests  that  the  key  to  determining  style 
effectiveness  is  the  extent  to  which  the  leader  can  or 
cannot  influence  followers  ( Sergiovanni  ,  1979).  This 
influence  will  be  the  result  of  three  situational 
variables :  (1)  leader-member  relations,  (2)  task  struc¬ 

ture,  and  (3)  position  power.  If  the  leader  finds 
himself  in  an  unfavorable  situation,  rather  than 
attempting  to  change  his  or  her  style  to  be  more 
effective,  the  leader  should  modify  the  situational 
variables  to  more  closely  match  his  or  her  style 
(Fiedler  ,  197*0. 

From  this  brief  synopsis  of  the  evolution  of  leader¬ 
ship  theory  it  becomes  evident  that  the  process  has 
involved  a  shift  in  perspective.  In  the  first  half  of 
the  century,  theorists  concentrated  entirely  on 
internal  traits,  attitudes,  and  behaviors  for  the 
answers.  In  the  second  half,  the  philosophy  has  shifted 
to  one  that  recognizes,  more  and  more,  the  importance 
of  environmental  factors,  such  as  the  leader's  sub¬ 
ordinates  and  work  situation,  in  ultimately  determining 


his  effecti veness . 
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Results  of  the  Empirical  Research 

The  ultimate  test  of  any  theory  is  the  success  of 
its  practical  application  to  the  real  world.  Since  the 
first  publishings  of  empirical  research  on  leadership  in 
1 9 0 14  ,  theorists  have  spent  exhaustive  efforts  attempting 
to  support  their  theories  with  scientific  results.  A 
major  impetus  to  the  field  occurred  during  World  War  I 
with  the  sudden  importance  in  the  selection  and  place¬ 
ment  of  military  officers.  Since  that  time  the  over¬ 
whelming  part  of  leadership  research  has  been  conducted 
in  the  U.S.,  and  the  field  continues  to  be  dominated  by 
U.S.  researchers.  The  primary  reasons  appear  to  be  the 
financial  support  provided  by  this  country  and  its 
democratic  ideology  which  enables  anyone  to  rise  to  a 
position  of  leadership  (Fiedler,  1971*). 

The  research  was  examined  for  evidence  that  sup¬ 
ported  the  claims  of  the  previously  discussed  theories, 
with  particular  emphasis  on  their  effects  on  group 
performance.  It  was  then  examined  for  proof  that 
leadership  training  could,  in  fact,  result  in  measur¬ 
able  improvement  in  leadership  effectiveness. 

As  mentioned  earlier,  years  of  research  by  the  trait 
theorists  created  lengthy  lists  covering  a  wide  range  of 


personality  attributes  but  little  consistency  across 
studies.  Stogdill  (197*0  found  only  three  traits  con¬ 
sistently  high  in  many  studies:  intelligence,  initiative 
and  responsibility.  If  leadership  effectiveness  was  due 
to  traits,  an  effective  leader  should  be  good  across  all 
kinds  of  situations.  Unfortunately  ,  we  know  from  real 
life  that  this  is  not  the  case. 

Research  on  behaviors  as  the  key  also  failed,  and 
efforts  shifted  to  styles.  There  has  been  a  great  deal 
of  research  aimed  at  determining  which  leadership  style 
is  the  most  effective.  Unfortunately ,  the  results  have 
been  quite  varied  and  depend,  to  a  great  deal,  upon  who 
conducts  the  study. 

As  mentioned  in  the  previous  section,  several 
theories  are  based  on  measuring  the  leader's  behavior  in 
■terms  of  his  or  her  orientation  towards  consideration  or 
the  initiation  of  structure.  Examining  the  effects  of 
this  emphasis  on  the  group  and  its  performance  reveals 
mixed  results . 

A  consensus  of  studies  show  a  high  positive  cor¬ 
relation  of  consideration  to  employee  satisfaction. 

While  not  surprising,  it  should  be  pointed  out  that  this 
relationship  speaks  nothing  of  causality.  An  employee 
may  well  become  more  satisfied  when  the  leader's 
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consideration  increases.  On  the  other  hand,  it  is 
entirely  possible  that  a  leader  tends  to  become  more 
considerate  with  a  productive,  satisfied  group 
(Fiedler  ,  1 97*0  . 

On  the  subject  of  productivity,  structure  has  a 
higher  positive  correlation  than  consideration.  Studies 
by  Vroom  (i960)  indicate  employee  characteristics 
determine  which  style  is  more  effective.  Employees  high 
in  authoritarianism  and  low  in  the  need  for  independence 
performed  better  under  more  structured  leaders,  while 
employees  high  in  the  need  for  independence  and  low 
authoritarianism  performed  better  under  more  considerate 
leaders.  Group  cohesiveness  appears  to  be  related  about 
equally  to  consideration  and  structure  in  the  positive 
direction  (Stogdill,  1974). 

Research  dealing  with  Hersey  and  Blanchard's 
Situational  Leadership  Model  fails  to  give  strong  support 
to  their  theory.  Twenty-six  studies  aimed  at  connecting 
leadership  style  and  effectiveness  found  no  consistent 
positive  correlation  (Blake  &  Mouton,  1982).  Their 
prescriptive  model  for  applying  situation  leadership 
theory  has  conceptual  inconsistencies,  particularly  in 
the  middle  range  levels  of  maturity  (M-2  and  M-3).  The 
validity  of  the  LEAD  diagnostic  instrument  for  measuring 


leader  style,  style  range,  and  effectiveness  is  suspect 
due  to  the  seemingly  arbitrary  way  Hersey  and  Blanchard 
decided  which  style  was  most  effective  in  several 
situations  (Graeff,  1983). 

If  one  collectively  groups  Blake  and  Mouton’s 
Managerial  Grid,  McGregor's  Theory  Y,  and  several  other 
similar  theories  together  as  participative  management, 
there  are  a  wide  variety  of  students  available  to  research 
and  analyze.  As  mentioned  previously,  high  consideration 
in  most  cases  increases  satisfaction  and  high  structure 
usually  increases  productivity.  Several  studies  indi¬ 
cate  that  high  levels  of  both  factors,  as  typified  by 
participative  management,  interact  to  influence  both 
productivity  and  satisfaction.  For  this  reason, 

Stodgill  (1974)  states  that  the  most  effective  leaders 
tend  to  be  described  high  on  both  scales  fitting  the  9,9 
style  in  terms  of  the  Managerial  Grid.  Blake  and 
Mouton  (1982)  cite  several  studies  where  large  corpora¬ 
tions  have  significantly  improved  productivity  by 
switching  the  "9,9’'  style  of  management.  Probably  the 
greatest  shortcoming  of  participative  management  is  its 
lack  of  consideration  for  the  uniqueness  of  the  situation. 
Its  main  thrust  is  on  what  leaders  do  to  followers,  not 
necessarily  what  followers  do  to  leaders  (Beck,  1982). 


IB 
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It  appears  that  Fiedler's  Contingency  Model  is  one 
of  the  best  validated  leadership  theories  today.  The 
Least  Preferred  Coworker  (LPC)  test,  a  personality  meas¬ 
ure,  is  the  key  variable  in  the  theory.  This  instrument 
has  shown  high  internal  consistency  and  test -re test 
reliability.  Using  a  leader's  LPC  score  and  a  descrip¬ 
tion  of  his  or  her  situational  favorableness,  the  theory 
has  demonstrated  that  it  is  highly  predictive  of  leader 
effectiveness  in  over  thirty  studies.  These  studies  have 
repeatedly  shown  that  low  LPC  leaders  (task-motivated) 
perform  best  in  situations  where  their  power  and  influence 
are  either  very  high  or  low.  High  LPC  leaders  (relation¬ 
ship-oriented)  perform  best  in  situations  where  their 
power  and  influence  are  moderate  (Fielder,  197*0.  The 
Contingency  Theory  is  not  without  its  shortcomings,  how¬ 
ever.  One  is  its  lack  of  specific  guidance  for  someone 
who  scores  in  the  mid-range  on  the  LPC  test  (Csoka  & 

Sons,  1978). 

What  evidence  does  the  research  give  to  support  the 
claims  that  leadership  training  increases  effectiveness? 
Unfortunately,  not  much,  and  much  of  what  has  been  done 
was  of  poor  design. 

Many  of  the  present  leadership  training  programs 
are  geared  around  sensitivity  training  which  concentrates 


on  group  dynamics  with  a  strong  human  relations 
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orientation.  In  general,  it  appears  that  this  type  of 
training  does  change  attitudes  in  the  desired  direction, 
towards  a  greater  relationship  awareness.  However,  there 
is  little  conclusive  evidence  that  this  training  results 
in  change  of  overt  behavior.  Also,  although  the  effects 
of  this  training  tend  to  increase  the  cohesive ness  of 
the  group,  it  is  often  associated  with  a  decrease  in 
group  productivity  (Stodgill,  1974). 

Many  of  the  training  programs  claim  that  they  can 
change  one's  leadership  style.  However,  the  results 
tend  to  indicate  that  leadership  styles  are  manifesta¬ 
tions  of  our  personalities,  and  as  such,  are  extremely 
difficult  to  change  ( Sergio vanni  .  1979  ). 

It  also  appears  that  the  effects  of  training  will  be 
dependent  upon  a  leader's  task  or  relationship-orientation. 
Some  evidence  supports  the  premise  that  under  certain 
circumstances  further  training  can,  in  fact,  hinder  a 
leader's  abilities.  The  Contingency  Theory  provides 
quite  accurate  guidelines  in  predicting  whether  such 
training  will  be  beneficial  or  detrimental  to  performance. 

Fiedler's  Leader  Match  training  program  is  one  of 
the  few  that  has  been  able  to  demonstrate  measurable 
improvements  in  leader  effectiveness  fairly  consistently, 
as  well  as  its  highly  desirable  attribute  of  cost 


effectiveness.  It  has  shown  significant  results  while 
requiring  less  than  twelve  hours  of  total  training 
(Leister,  Borden  &  Fiedler,  1977).  While  this  program 
seems  to  work,  part  of  its  success  may  be  due  simply 
to  its  effects  as  a  confidence  builder.  In  several 
studies  attendees  showed  improved  performance  but 
demonstrated  a  lack  of  understandi ng  of  the  techniques 
involved  in  changing  the  situational  variables  (Csoka 
&  Eons,  1978). 

Overall,  it  appears  that  leadership  training  can  be 
effective  in  certain  situations.  However,  measurable 
results  are  often  difficult  to  show.  Training  programs 
that  male  strong  guarantees  of  success  and/or  style 
changes  should  be  viewed  with  skepticism.  In  many 
cases,  leadership  training  is  not  needed  as  much  as 
changes  in  organizational  practices  and  managerial  skills 
that  tend  to  restrict  employees'  performance  and  pro¬ 
ductivity  (Dreilinger,  et  al.,  1982).  For  this  reason, 
many  of  today's  leadership  programs  incorporate  train¬ 
ing  in  such  techniques  as  decision  making,  conflict 
resolution,  group  facilitation,  etc.  It  appears  that 
mastering  these  managerial  skills  may  be  as  critical  as 
understanding  the  latest  leadership  theories. 


Program  Evaluation 


How  do  we  determine  if  a  particular  leadership 
training  program  is  effective;  i.e.,  that  as  a  result 
of  the  change  caused  in  the  attendee,  his  or  her  group 
is  able  to  perform  their  assigned  functions  better? 

This  involves  program  evaluation,  which  has  developed 
into  a  completely  independent  field  of  study. 

The  term  "program"  is  extremely  broad  in  scope 
and  can  fit  a  variety  of  descriptions.  However,  if  we 
limit  our  discussion  to  educational  programs,  these  can 
be  defined  as  a  combination  of  content,  personnel, 
activities,  and  resources  organized  so  as  to  attain 
specified  goals  and  objectives  (California  State 
Department  of  Education,  1979). 

Even  without  this  context,  program  evaluation  can 
be  conducted  for  numerous  reasons.  Four  major  purposes 
for  program  evaluation  are: 

(1)  Communicating  with  the  public; 

(2)  Providing  information  to  decision  makers; 

(3)  Improving  existing  programs; 

(4)  Generating  a  sense  of  unity  and  growth. 

Just  as  there  are  many  reasons  for  conducting  an 

evaluation,  the  process  itself  can  be  accomplished 
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using  a  variety  of  methods.  There  are  a  great  number  of 
books  published  today  that  serve  as  guides  for  this 
process.  Most  agree  that  the  evaluation  process 
involves  three  major  phases:  planning,  conducting,  and 
using.  The  first,  planning,  involves  determining  the 
evaluation's  purposes  and  objectives.  From  this  one 
can  develop  a  general  evaluation  plan  which  includes 
determining  its  design  and  obtaining  the  necessary 
assessment  tools.  The  second  phase,  conducting,  involves 
both  the  collection  and  analysis  of  the  required  data. 

The  third  and  final  phase,  using,  involves  reporting  the 
results  and  applying  the  evaluation  findings  to  future 
recommendations  (California  State  Department  of  Educa¬ 
tion,  1979). 

Probably  the  most  critical  and  yet  most  neglected 
phase  is  the  initial  one.  Without  proper  planning, 
tremendous  amounts  of  effort  and  money  can  be  expended 
with  little  hope  of  obtaining  meaningful  data  from  which 
significant  conclusions  can  be  drawn.  The  California 
State  Department  of  Education  (1979)  has  developed  a 
fairly  comprehensive  checklist  for  guidance  through 
the  initial  steps  of  defining  the  evaluation's  purposes 
and  requirements  (see  Figure  1). 

Developing  the  evaluation  plan  depends  to  a  great 


DEf INE  L V A L U A U 0 N  PURPOSE 


•  Determine  from  decision  makers  the  purpose  of  the  evaluation.  The  purpose  will 
dictate  the  tyoes  of  evaluation  that  must  be  conducted. 

REVIEW  NEEDS  ASSESSMENT,  PROGRAM  GOALS,  AND  PROGRAM  OBJECTIVES 

•  Determine  whether  a  needs  assessment  was  conducted  by  the  decision  makers. 

t  Review  program  goals  to  determine  whether  they  address  the  needs  of  problem  ere8s. 

•  Review  performance  objectives  to  determine  that  they  sre  compatible  with  program 
goals.  Are  the  objectives  stated  in  unambiguous  terms? 

Rf-iEW  PROGRAM  ACTIVITIES 

«  Review  program  activities  to  judge  whether  they  can  be  expected  to  contribute  to 
achievement  of  the  objectives. 

e  If  the  activities  do  not  match  the  objectives,  recommend  that  activities  or 
objectives  be  revised. 

identity  evaluation  requirements 

e  Reouest  that  decision  makers  identify  the  information  they  will  require  to  make 
er.d-of-year  decisions  about  the  program. 

•  Determine  process,  product,  and  context  data  that  should  Pe  collected. 

e  Determine  the  information  required  by  decision  makers  to  make  interim  decisions. 

t  Determine  when  the  information  is  required. 

IDENTITY  evaluation  RESOURCES  and  constraints 

t  Determine  the  resources  and  constraints  which  will  affect  the  conduct  of  the 
evaluation . 

•  Advise  decision  makers  of  those  resources  which  are  available  and  those  that  are 
required- 

o  Submit  recommendations  to  decision  makers  for  reconciling  discrepancies  between 
resources  available  and  those  required. 


Fiqure  1 

Checklist  of  the  Steps  in  Determining  Evaluation  Purposes 
Requirements 


California  State  Department  of  Education,  Program  Evaluator's  Guide  (Princeton: 
Educational  Testing  Service,  T979 - 
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extent  upon  the  purposes  defined  for  the  evaluation. 
Hoyle,  English,  and  Steffy  (1985)  describe  various 
methods  that  can  be  used  to  accomplisn  the  appropriate 
objectives  of  the  evaluation  (see  Figure  2). 

If  the  planning  phase  has  been  completed  properly, 
the  conducting  phase  becomes  fairly  straight  forward. 

The  specific  steps  of  data  collection  and  analysis  are 
well  defined  within  the  guidelines  of  educational 
research  and  primarily  involve  statistical  techniques. 

The  final  phase,  using  the  results  obtained,  again 
depends  upon  the  initial  purposes  of  the  evaluation. 

In  most  cases,  though,  the  evaluation  process  should  not 
end  with  this  phase,  but  continue  as  an  ongoing  process. 
The  findings  and  results  should  be  used  to  make  decisions 
and  possible  modifications  to  the  present  program  which, 
in  turn,  will  need  re-evaluation. 
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TYPE 

DESCRIPTION 

PROTOGONISTS 

Student  Gain  by 
Testing 

To  measure  student  performance  and 
progress.  Goal  statements  need  to  be 
specified;  test  score  analysis  of 
discrepancy  between  goal  and  actuality. 

Ralph  Tyler 

Benjamin  Bloom 

James  Popham 

Mai  Provus 

Instituior. al  Self- 
study  by  Staff 

To  review  and  increase  staff  effectiveness. 
Committee  work  standards  set  by  staff; 
discussion 

National  Study 
of  School 

E  v  al  uation 

Dressel 

Blue  Ribbon  Panel 

To  resolve  crises  and  preserve  the 
institution.  Prestigious  panel;  the 
visit;  review  of  existing  data  and 
documents. 

James  Conant 

Clark  Kerr 

David  Henry 

Transaction 

Observation 

To  provide  understanding  of  activities 
and  values 

Louis  Smith 
Parlett-Hamilton 
Robert  Rippey 

Robert  Stake 

Management 

Analysis 

To  increase  rationality  in  day-to-day 
decisions.  Lists  of  options; 
estimates;  feedback  loops;  costs; 
efficiency 

Leon  Lassinger 
Daniel  Stuffiebeam 
Marvin  Alkin 

Alan  Thomas 

Instructional 

Research 

To  generate  explanations  and  tactics 
of  instruction. 

Controlled  conditions, 
multivariate  analysis;  bases  for 
generalization 

Lee  Cronbach 

Julian  Stanley 
Donald  Campbell 

Social  Policy 
Analysi  s 

To  aid  development  of  institutional 
policies. 

Measures  of  social  conditions  and 
administrative  implementation. 

James  Coleman 

David  Cohen 

Carol  Weiss 
Mosteller-Maynihan 

CHAPTER  III 


METHODOLOGY 

Design 

The  Basic  Supervisory  Development  (BSD)  Seminar  is 
a  course  that  has  been  developed  to  meet  some  general 
guidelines  set  by  the  Naval  Civilian  Personnel  Command. 
The  course's  instructional  objectives  are  stated  in  a 
general  nature  and  no  requirements  are  made  of  attendees 
which  demonstrate  successful  mastering  of  the  material 
presented.  Attendance  is  the  sole  requirement  for 
completion  of  the  course.  These  factors  limited  the 
extent  to  which  this  program  evaluation  could  be  con¬ 
ducted  under  the  strict  parameters  discussed  in  Chapter 
II.  However,  it  was  felt  that  meaingful  information 
could  still  be  obtained  if  the  proper  design  was 
utilized.  With  this  in  mind,  the  study  was  designed  to 
meet  two  of  the  four  major  purposes  for  program  evalu¬ 
ation  discussed  in  Chapter  II: 

(1)  Provide  information  to  decision  makers; 

(2)  Improve  existing  programs. 

Phase  I  of  the  study  involved  descriptive  research 
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designed  to  measure  attitudes  of  past  attendees  towards 
the  effectiveness  and  adequacy  of  the  course  content. 

The  resulting  data  could  be  used  as  guidelines  for 
future  modification  of  course  emphasis  or  content. 

Phase  II  involved  an  experimental  design  with  the  Basic 
Supervisory  Development  Seminar  as  the  treatment  or  inde¬ 
pendent  variable,  and  the  leadership  styles  of  the 
attendees  as  the  dependent  variable.  This  resulted  in 
the  following  null  hypothesis: 

There  will  be  no  significant  change  in  the 
leadership  styles  of  people  who  attend  the 
BSD  Seminar. 

Here,  again,  the  results  could  be  used  to  determine 
if  the  course  was  having  the  desired  effect  on  its 
attendees  or  whether  re-direction/modification  was 
necessary . 


Subjects 


All  participants  in  this  study  were  federal  employees 
of  the  Naval  Civilian  Personnel  Command  recently  promoted 
to,  or  soon  to  be  eligible  for,  a  first-line  supervisory 
position.  Both  the  sample  groups  and  the  target  popula¬ 
tion  consisted  of  males  and  females  from  multi-ethnic, 
backgrounds  ranging  in  age  from  the  mid-twenties  to 


mid-forties.  The  sample  group  (90  subjects)  for  Phase  I 
consisted  of  all  attendees  of  the  BSD  Seminars  held 
from  January  1984  to  February  1985.  Phase  II  involved 
63  subjects,  comprised  of  attendees  from  three 
different  sessions  of  the  course  held  in  May  and  June  of 
1985. 

Instruments 

Based  upon  seminar  attendance  and  course  outline 
handouts,  a  personally  designed  questionnaire  was  used 
in  Phase  I  to  survey  the  attitudes  of  past  seminar 
attendees  (See  Appendix  A).  The  course  content  was 
divided  into  six  major  topics.  The  questionnaire 
utilized  a  five-point  Likert  scale  to  measure  the  extent 
to  which  the  attendees  felt  these  six  topics  were 
adequately  covered,  from  "Requires  More  Emphasis"  to 
"Requires  Less  Emphasis."  The  attendees  were  also 
asked  to  indicate  which  single  topic  had  the  greatest 
impact  on  their  leadership  style /supervisory  effective¬ 
ness  . 

The  questionnaire  was  assessed  by  the  seminar's 
developer /instructor  and  an  education  department  pro¬ 
fessor  for  its  content  validity.  It  was  pre-tested  on 
a  small  pilot  group  to  measure  its  readability  and  the 
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clarity  of  its  instructions.  The  actual  sample  group 
was  restricted  to  those  who  had  attended  the  seminar 
within  the  past  year  in  a'n  attempt  to  assure  reasonable 
familiarization  with  the  course  content. 

For  Phase  II,  Pfeiffer  and  Jones'  "T-P  Leadership 
Questionnaire"  (see  Appendix  B)  was  used  to  obtain  a 
pre-  and  post-test  measurement  of  the  attendees' 
leadership  styles.  This  questionnaire  was  adapted  from 
the  "Leadership  Behavior  Description  Questionnaire, 

Form  XII"  of  the  well  known  Ohio  State  Leadership 
Studies  ( Sergiova.nni  ,  Metzcus  &  Burden,  1969  ).  Dis¬ 
cussions  with  Dr.  Solomon,  the  course  developer,  were 
a  key  factor  in  the  selection  of  this  particular  instru¬ 
ment.  One  of  the  stated  goals  of  the  seminar  is  to 
encourage  a  participative  style  of  leadership  which  is 
characterized  by  a  high  concern  for  both  people  and 
task.  The  T-P  Leadership  Questionnaire  provides  a 
measurement  of  these  two  variables  in  a  relatively  short 
and  easy- to-adminis ter  test  format.  Respondents  are 
required  to  indicate  hew  they  would  react  to  35  items 
which  describe  various  aspects  of  leadership  behavior. 
From  their  responses  scores  are  obtained  that  indicate 
their  concern  for  people  and  their  concern  for  task. 
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Procedure 

For  Phase  I  ail  past  attendees  of  the  BSD  Seminar 
from  January  'i  9  £  ^  to  February  1  985  were  mailed  the  BSD 
Cues r i cnnaire  along  with  a  cover  letter  (see  Appendix  C) 
a. no  a  rr e-addressed  ,  stamped,  return  envelope.  The 
a imi nistrati ve  staff  of  the  Naval  Civilian  Personnel 
Command.  Southwest  Region's  Training  Department  per¬ 
formed  this  function,  thus  enabling  use  of  the  Department 
of  the  Navy's  letterhead  and  postal  privileges.  The 
staff  retained  all  returned  questionnaires  until  they 
were  collected  by  the  researcher  for  data  analysis. 

Phase  II  involved  administering  the  Leadership 
Cues t ionnai re  (see  Appendix  B)  to  several  sessions  of 
the  seminar  before  and  after  attending  the  course . 

While  use  of  the  same  measurement  for  pre-  and  post- 
tests  introduced  testing  as  a  threat  to  the  internal 
validity  cf  this  phase,  this  shortcoming  was  outweighed 
:y  the  necessity  of  controlling  for  two  other  threats 
to  internal  validity.  A  pre-test  could  be  used  to 
determine  initial  differences  between  groups  thus  limit¬ 
ing  the  effects  of  differential  selection  of  subjects, 
and  could  also  be  used  to  control  for  mortality  in  the 
event  that  not  all  pcst-tests  were  returned. 


Due  to  the  irregularity  of  the  scheduled  sessions 


and  the  time  constraints  of  this  project,  only  three 
sessions  were  available  for  evaluation.  For  each,  the 
procedure  remained  the  same.  On  the  first  morning  of 
the  seminar,  prior  to  starting  his  presentation,  the 
instructor  read  aloud  the  Pre-Test  Instructions  (see 
Appendix  D)  and  then  distributed  the  questionnaires  for 
completion.  After  completion,  the  tests  were  collected 
and  retained  by  the  instructor  for  later  turnover  to 
the  researcher  for  data  analysis. 

Administering  the  post-test  presented  several 
problems.  Attendees  came  from  various  commands  through¬ 
out  southern  California  and  were  together  as  a  group 
only  for  the  duration  of  the  seminar.  It  was  felt  that 
a  post-test  measurement  given  immediately  upon  comple¬ 
tion  of  the  course  would  not  represent  actual  long-term 
effects  of  the  seminar.  It  was  reasoned  that  a  more 
accurate  measure  of  any  real  changes  could  be  obtained 
by  administering  a  post-test  one  month  later.  This 
delay  would  have  three  advantages:  (1)  it  would  allow 
time  for  the  novelty  effect  of  the  course  to  wear  off; 

(2)  it  would  give  the  attendees  time  to  experience  some 
on-the-job  applications  of  the  material  presented;  and 

(3)  at  the  same  time  it  would  reduce  the  possible  effect 


27 


I 

■ 


r 


of  being  re-tested  with  the  same  test  too  soon. 

Because  it  was  impossible  to  re-form  the  group  to 
administer  the  post-test,  a  mailing  process  similar  to 
the  one  in  Phase  I  was  used.  Three  weeks  after  the 
completion  date  of  the  course  a  copy  of  the  Leadership 
Questionnaire,  along  with  a  cover  letter /instructions 
(see  Appendix  E)  and  a  pre-addressed ,  stamped,  return 
envelope  were  mailed  to  the  home  of  each  attendee. 
Allowing  several  days  for  mail  delivery  and  several  more 
until  the  test  would  probably  be  taken,  the  post-test 
should  have  been  taken,  as  intended,  approximately  one 
month  after  course  completion. 

One  difference  in  the  mailing  process  between  Phase 
I  and  II  should  be  noted.  In  Phase  I  the  questionnaires 
were  mailed  to  the  training  officer  of  the  individual 
commands  to  be  distributed  by  them  to  the  attendees  in 
the  most  convenient  manner.  Depending  on  the  size  of 
the  command,  they  could  have  been  hand  delivered,  routed 
administratively,  or  mailed.  In  Phase  II  the  question¬ 
naires  were  mailed  directly  to  the  homes  of  the 
attendees  rather  than  their  work  place.  This  was  done 
to  simplify  the  process  and  to  meet  project  completion 
deadlines . 

The  scoring  procedure  for  both  the  pre-  and 
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post-tests  was  modified  slightly  from  that  specified  in 
Pfeiffer  and  Jones  (197*0.  (See  Appendix  F.)  To  prevent 
the  attendees  from  becoming  aware  of  the  purpose  of  the 
instrument,  both  the  pre-  and  post-tests  were  scored  by 
the  researcher  rather  than  the  attendees. 

Limitations 

There  were  several  uncontrollable  factors  that  must 
be  recognized  as  limitations  to  this  study.  One  is  the 
fact  that  the  course  content  of  the  BSD  seminar  varies 
somewhat  with  location.  When  the  seminar  is  given  in 
San  Diego  it  consists  of  five  days  of  instruction.  The 
first  day  is  given  by  Dr.  Phillip  Hunsaker  and  the 
remaining  four  by  Dr.  Lawrence  Solomon.  When  the  seminar 
is  given  outside  of  the  San  Diego  area  it  is  given  in  a 
•four-day  version  solely  by  Dr.  Solomon.  Dr.  Hunsaker's 
material  is  separate  but  similar  to  Dr.  Solomon's. 

Dr.  Hunsaker  presents  an  introduction  to  management 
principles,  leadership  styles,  and  leadership  skills. 
Practically  all  of  this  information  is  covered  with 
additional  detail  by  Dr.  Solomon.  While  the  overall 
course  content  of  both  versions  of  the  seminar  can  be 
assumed  to  be  essentially  the  same,  the  five-day  format 
must  be  considered  to  be  a  more  in-depth  version  which 
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exposes  the  attendees  to  an  additional  expert  on  the 
subject  matter.  Of  the  three  sessions  evaluated  in 
Phase  II,  Class  I  was  of  the  five-day  format,  while 
Classes  II  and  III  were  of  the  shorter  version. 

Unfortunately,  it  was  not  feasible  to  include  a 
control  group  into  the  design  of  Phase  II  of  the  project. 
All  employees  who  advance  into  a  supervisory  position 
eventually  attend  the  seminar.  Therefore,  a  group  of 
supervisors  who  had  not  attended  the  course  could  not  be 
identified  for  comparison  purposes.  This  resulted  in 
use  of  a  pre-  and  post-test  design.  However,  as  pre¬ 
viously  discussed,  it  was  impossible  to  administer  the 
post-test  under  conditions  identical  to  the  pre-test. 

The  post-test  instructions  requested  that  the  attendee 
attempt  to  take  the  test  in  the  morning  under  similar 
conditions  to  the  pre-test,  but  ther-^  is  no  way  to 
determine  if  that  actually  occurred. 

Also,  as  previously  mentioned,  due  to  the  time  con¬ 
straints  involved,  only  three  sessions  could  participate 
in  Phase  II.  The  size  of  these  sample  groups  were  27, 

18,  and  18  attendees.  The  number  of  those  attendees  who 
returned  a  completed  post-test  were  19,  10,  and 
respectively.  The  lack  of  necessary  post-test  data  for 
the  third  group  dictated  elimination  of  this  group  from 


statistical  consideration. 


CHAPTER  IV 


RESULTS 

Phase  I 

Ninety-five  Basic  Supervisory  Development  Ques¬ 
tionnaires  (see  Appendix  A)  were  mailed  to  past 
attendees  of  the  course.  Five  of  these  were  returned 
unanswered  because  four  people  had  not  actually  attended 
the  course  and  one  person  had  since  retired  and  was 
unreachable.  Of  the  remaining  90  questionnaires,  83 
were  completed  and  returned  for  a  participation  rate  of 
52. 2  percent.  The  demographic  questions  indicated  that 
the  sample  group  consisted  of  69.9  percent  males  and 
that  72.3  percent  of  the  83  attendees  were  currently 
working  in  a  supervisory  position. 

The  frequencies  of  responses  to  the  question  of 
adequate  coverage  of  the  six  major  topics  were  compiled 
and  tabulated  into  Table  1 .  The  five  columns  represent 
the  percentage  of  the  total  responses  for  each  category 
from  "Requires  More  Emphasis"  on  the  far  left,  to 
"Requires  Less  Emphasis"  on  the  far  right. 

From  Table  1  it  is  evident  that  a  majority  of  past 
attendees  feel  that  the  present  course  content  provides 
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Table  1 

Phase  I  Questionnaire  Responses 


Topic  No 


More 

Emphasis 


Adequate 

Coverage 


Less 

Emphasis 


1 

2.4 

7.2 

74.7 

7.2 

8.4 

2 

19.3 

18.1 

60.2 

1 .2 

1  .2 

3 

10.8 

25.3 

60.2 

3.6 

0 

4 

1 8 . 1 

19.3 

56.6 

6.0 

0 

5 

13.3 

24.1 

53.0 

9.6 

0 

6 

21  .7 

28.9 

43.4 

4.8 

1  .2 

Table  1  represents  the  frequency  of  responses  in 
percentages  for  the  six  major  topics  of  the  seminar  as 
taken  from  the  BSD  Questionnaire  (see  Appendix  A).  For 
Topics  Nos.  1  through  5  a  clear  majority  felt  that  the 
present  coverage  of  the  topic  was  adequate.  For  Topic 
No.  6,  if  the  two  degress  of  "More  Emphasis"  are 
combined,  a  slight  majority  of  50.6  percent  indicated  a 
desire  for  more  emphasis  of  that  subject  matter. 

Topic  No.  l  was  the  only  topic  which  had  a  greater 
combined  percentage  of  responses  desiring  less  coverage 
than  more. 
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adequate  coverage  of  the  first  five  topics.  Combining 
the  two  degress  of  "More  Emphasis"  columns  for  Topic  No. 

6  shows  that  a  slight  majority  (50.6%)  feel  more  emphasis 
is  needed  on  Interviewing  and  Counseling.  Topic  No.  1, 
Historical  Background  and  Management  Theories,  was  the 
only  topic  which  had  a  greater  percentage  of  the  respon¬ 
dents  requesting  less  coverage  rather  than  more  coverage. 

On  the  question  of  which  major  topic  has  had  the 
greatest  impact  on  the  attendee's  leadership  style/ 
supervisory  effectiveness,  only  69  out  of  the  83  respon¬ 
dents  properly  indicated  a  single  topic.  These  results 
were  tabulated  by  frequency  of  responses  and  corres¬ 
ponding  percentages  into  Table  2.  Using  these  data,  a 
one-dimensional  chi-square  was  used  to  determine  if  the 
observed  frequencies  were  significantly  different  from 

what  could  be  expected  by  chance  alone.  The  results  of 

2 

this  test  were  significant  (X  =  25.7  §  p  =  .01  with 
df  =  5). 

Table  2  indicates  that  every  topic  had  at  least  one 
response.  However,  only  two  topics,  Nos.  2  and  were 
markedly  above  what  you  would  expect  by  chance,  16.7 
percent.  The  topic  with  the  greatest  number  of  responses, 
No.  t ,  was  Decision-making  and  Problem-solving.  Topic 
No.  2  dealt  with  the  Supervisor  as  Facilitator. 
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Table  2 

Topic  of  Greatest  Impact 


Topic  No. 

1 

2 

3 

4  5 

6 

Responses 

1 

17 

13 

22  5 

1  1 

Percentage 

1  .5 

2M  .6 

18 

.8  31.9  7.3 

15.9 

Table  2  represents  the  frequency  of  responses 
and  corresponding  percentages  for  the  question  of  which 
seminar  topic  had  the  greatest  impact  on  the  attendee's 
leadership  style /supervisory  effectiveness.  There  was  a 
significant  difference  between  the  observed  responses 
and  what  would  be  expected  by  chance  alone,  16.7  percent, 
with  Topics  Nos.  2  and  9  receiving  a  markedly  greater 
percentage  of  those  responses  . 


On  the  open-ended  "Comments"  section  of  the  ques¬ 
tionnaire,  38  respondents  (45.8%)  made  some  type  of 
comment.  Of  these,  approximately  half  were  strictly 
laudatory  comments  directed  at  the  course  and/or 
Dr.  Solomon.  The  remaining  half  were  specific  recom¬ 
mendations  mostly  geared  towards  areas  of  the  course 
that  in  their  opinion  required  more  or  less  emphasis. 

Phase  II 

Pre-Test  to  Pre-Test  Comparison: 

As  discussed  in  Chapter  III,  lack  of  returned 
post-tests  presented  a  problem  in  this  phase  of  the 
project.  For  this  reason,  Class  III,  with  only  four  of 
18  post-tests  returned,  was  eliminated  from  statistical 
consideration .  In  addition  several  statistical  tech¬ 
niques  were  used  to  determine  if  the  lack  of  returned 
post-tests  affected  the  results  of  the  other  two  classes. 
For  both  classes  t-tests  for  independent  samples  were 
used  to  compare  the  pre-test  scores  of  those  who  did 
not  return  a  post-test  with  those  who  did  return  one. 

On  Tables  3  and  4  the  upper  portion  of  the  table  repre¬ 
sents  the  "people"  and  "task"  scores  for  those  attendees 
who  did  not  return  a  test, 


and  the  lower  portion 
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Table  3 

Class  I:  Pre-Test  Scores 


Attendees  Who  Did  Not  Return  a  Post-Test 
Pre-Test  People  Pre-Test  Task 


Mean 


13.3 


2.23 


3.11 


Attendees  Who  Did  Return  a  Post-Test 
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Table  3  (continued) 


Attendees  Who  Did  Return  a 
Pre-Test  People 

Post-Test 

Pre-Test  Task 

10 

18 

9 

9 

u 

1 1 

15 

9 

13 

9 

12 

7 

18 

6 

9 

9 

10 

1U 

10 

10 

Mean  9.2 

13.2 

SD  1.63 

3.3B 

Table  3  compares  the  pre-test  scores  as  measured  by 
the  Leadership  Questionnaire  (see  Appendix  B)  Tor 
Class  I  of  those  who  did  not  return  a  post-te3t,  with 
those  who  did.  There  was  no  significant  difference 
between  the  means  for  either  group's  People  or  Task 
scores . 
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Table  4 


Class  II: 

Pre-Test  Scores 

Attendees  Who  Did 
Pre-Test  People 

Not  Return 

a  Post-Test 

Pre-Test  Task 

12 

5 

12 

12 

5 

5 

10 

10 

9 

12 

12 

3 

10 

1 1 

Mean 

9.9 

8.8 

SD 

2.2 

3.53 

Attendees  Who 

Did  Return 

a  Post-Test 

12 

14 

1 1 

10 

i0 

13 

7 

15 

1 1 

13 

5 

12 

1 1 

10 

9 

14 

1 1 

12 

7 

4 

Mean 

9.4 

1 1  .7 

SD 

2.2 

3-0 

Table  4  compares  the  pre-test  scores  as  measured  by 
the  Leadership  Questionnaire  (see  Appendix  B)  for  Class 
II  of  these  attendees  who  did  not  return  a  post-test  with 
those  who  did.  There  was  no  significant  difference 
between  the  means  of  either  group's  People  or  Task  scores . 


represents  the  scores  of  those  who  did  return  one.  As 
Figure  3  indicates,  for  both  classes,  in  all  cases,  the 
t-tests  failed  to  show  a  significant  difference  between 
the  scores  of  those  attendees  who  returned  a  post-test 
and  those  who  did  not  return  one.  This  lends  statis¬ 
tical  support  to  the  argument  that  although  not  all 
post-tests  were  returned,  those  that  were  can  still  be 
considered  representative  of  the  group  as  a  whole. 


Class  I: 


Class  II: 


t  .  =0.652 
people 


Has^0-70 


§ p*  .01  and  df  =  25 


t  .  =0.45 
people 


W51-75 


§  p  =  .01  and  df  = 1 6 


Figure  3 

Results  of  Statistical  Comparison  of 
Pre-Test  Scores 


T-tests  for  independent  samples  were  calculated  to 
compare  the  scores  of  those  not  returning  post-tests 
with  those  who  did  return  post-tests.  In  all  four  cases 
there  was  no  significant  difference  found. 
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Pre-Test  to  Post-Test  Comparison: 

In  order  to  determine  if  the  difference  between  the 
pre-  and  post-test  scores  was  significant  or  not,  t-tests 
for  nonindependent  samples  were  run  on  both  classes. 
Tables  5  through  8  indicate  the  pre-  and  post-test  scores 
and  resulting  change  on  both  variables,  "people"  and 
"task"  concern.  Use  of  these  data  resulted  in  the  sta¬ 
tistics  represented  by  Figure  4.  In  all  four  cases 
there  was  a  significant  difference  between  the  pre-  and 
post-test  scores,  thus  supporting  rejection  of  the  null 
hypothesis . 


Class  I: 


t  .  =4.66* 
people 


§  p  =  .01  and  df=i8 


Class  II : 


t 


people 


=  3.54* 


'task 


=  4.67* 


@ p  =  .01  and  df  =  9 


♦Significant  value 


Figure  4 

Results  of  Statistical  Comparison  of  Pre-Test  to 

Post-Test  Scores 


T-tests  for  nonindependent  samples  indicate  that  in 
all  four  cases  there  was  a  significant  difference  between 
pre-  and  post-test  scores. 


Table  5 


Class  I :  People  Scores 


Pre-Test  People 

Post-Teat  People 

Change 

9 

14 

-5 

1  0 

5 

-5 

12 

1  0 

-2 

8 

7 

-1 

10 

9 

-1 

9 

8 

-1 

7 

6 

-1 

13 

1  3 

0 

1 1 

1 1 

0 

7 

7 

0 

6 

6 

0 

10 

10 

0 

9 

10 

1 

8 

10 

2 

9 

1  1 

2 

9 

1  1 

2 

9 

12 

3 

9 

12 

3 

9 

13 

14 

Mean  9.16 

9  .21 

1  .7H 

Table  5  represents  the  pre-  and  post-test  scores  and 
resultant  change  for  Class  I  on  their  "concern  for  people 
as  measured  by  the  Leadership  Questionnaire  (see  Appendix 
E).  While  the  overall  mean  only  slightly  increased  from 
the  pre-  to  the  post-test  scores,  the  difference  was 
significant  and  Table  5  indicates  that  an  equal  number  of 
attendees  experienced  an  increase  in  their  scores  as 


Table  6 
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Class  I:  Task  Scores 


Pre-Test  Task 

Post-Test  Task 

Change 

13 

4 

-9 

17 

10 

-7 

15 

10 

-5 

1  4 

9 

-5 

9 

6 

-3 

17 

1  4 

-3 

12 

10 

-2 

4 

2 

-2 

1 8 

17 

-1 

12 

1 1 

-1 

15 

14 

-1 

18 

17 

-1 

9 

9 

0 

1  4 

15 

1 

1  4 

15 

1 

1  0 

12 

2 

13 

16 

3 

1  4 

17 

3 

13 

19 

6 

Mean  13.2 

11.9 

2.95 

Table  6  represents  the  pre-  and  post-test  scores  and 
resultant  change  for  Class  I  on  their  "concern  for  task" 
as  measured  by  the  Leadership  Questionnaire  (see  Appendix 
B).  The  overall  decrease  in  the  mean  scores  was  signifi¬ 
cant  and  as  Table  6  indicates,  twice  as  many  attendees 
experienced  a  decrease  in  their  task  scores  as  those  that 
experienced  an  increase  in  scores. 
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Table  7 

Class  II:  People  Scores 


Pre-Test  People 

Post-Test  People 

Change 

1 1 

9 

-2 

12 

1 1 

-1 

1  1 

10 

-1 

5 

5 

0 

1  1 

1 1 

0 

7 

8 

1 

1  1 

12 

1 

9 

10 

1 

10 

12 

2 

7 

1  1 

4 

Mean  9.4 

9.9 

1  .3 

Table  7  represents  the  pre-  and  post-test  scores  and 
resultant  change  for  Class  II  on  their  "concern  for 
people"  as  measured  by  the  Leadership  Questionnaire  (see 
Appendix  B).  As  with  Class  I,  the  difference  in  the  mean 
scores  was  significant  and  in  the  positive  direction.  In 
this  class  more  attendees  experienced  an  increase  in  their 
people  scores  than  a  decrease  in  scores. 


Table  8 


Class  II:  Task  Scores 


Pre-Test  Task 

Post-Test  Task 

Change 

10 

4 

-6 

12 

7 

-5 

10 

6 

-4 

1  4 

1 1 

-3 

12 

10 

-2 

13 

13 

0 

15 

16 

1 

13 

15 

2 

14 

16 

2 

4 

6 

2 

Mean  11.7 

10.6 

2.7 

Table  8  represents  the  pre-  and  post-test  scores 
and  resultant  change  for  Class  II  on  their  "concern  for 
task"  as  measured  by  the  Leadership  Questionnaire  (see 
Appendix  E).  As  with  Class  I,  the  difference  in  the 
mean  scores  was  significant  and  in  the  negative 
direction.  There  was  also  more  attendees  who  experi¬ 
enced  a  decrease  in  their  scores  and  the  amount  of  that 
change  was  greater  than  those  who  experienced  an 
increase  in  scores. 


iu 

Further  examination  of  Tables  5  through  8  revealed 
several  tendencies.  For  both  classes  there  was  a  slight 
increase  in  the  people  soores  after  attending  the 
seminar.  At  the  same  time  there  was  a  decrease  in  the 
task  scores  for  both  classes  and  the  average  amount  of 
that  change  was  greater  for  task  scores  than  people 
scores . 

Post-Test  to  Post-Test  Comparison: 

In  the  interest  of  determining  whether  there  was  a 
significant  difference  between  the  post-test  scores  of 
the  two  classes,  t-tests  for  independent  samples  were 
run  using  the  post-test  scores  from  Tables  5  through  8. 
The  resulting  values,  *76  and  -838 ,  were 

not  significant  @  p=  ,01  and  df=27. 
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CHAPTER  V 

SUMMARY,  CONCLUSIONS,  AND  RECOMMENDATIONS 

Summary 

A  two-phase  program  evaluation  was  conducted  of  the 
Naval  Civilian  Personnel  Command's  Basic  Supervisory 
Development  Seminar.  Phase  I  involved  descriptive 
research  based  on  a  questionnaire  survey  of  past 
attendees  to  determine  their  attitudes  on  the  appropri¬ 
ateness  of  the  course  content.  Results  indicated  that 
the  emphasis  was  appropriate  on  all  but  one  of  the  six 
major  topics  of  the  seminar.  Phase  II  involved  a  pre¬ 
test/post-test  experimental  design  to  determine  whether 
the  seminar  was  having  a  significant  influence  on  the 
leadership  styles  of  its  attendees.  The  somewhat 
limited  results  obtained  did  show  a  significant  influence 
on  the  sample  groups  that  were  involved  in  this  part  of 
the  study. 


Conclusions 


Phase  I 

Several  conclusions  can  be  drawn  from  the  results 
of  Phase  I  of  the  program  evaluation. 


1 .  The  sample  group  was  highly  representative  of 
the  target  population.  If  we  consider  all  those  eligible 
for  the  BSD  Seminar  as  the  target  population,  the  per¬ 
centage  of  subjects  responding  to  the  survey,  the  time 
duration  involved,  and  the  demographic  percentages 
reported,  all  lend  strong  support  to  the  representative¬ 
ness  of  the  sample  group. 

2.  Based  on  the  experiences  and  opinions  of  past 
attendees  as  expressed  on  the  BSD  Questionnaire,  the 
emphasis  of  the  present  course  content  appears  to  be 
appropriate,  with  the  possible  exception  of  Topic  No.  6. 
It  appears  that  some  greater  emphasis  on  Interviewing 
and  Counseling  could  be  justified. 

3.  There  was  a  significant  difference  in  the 
responses  on  the  topic  of  greatest  impact  with  Topic 

No.  4,  Decision-making  and  Problem-solving,  being  chosen 
most  often.  However,  the  varied  distribution  of  those 
responses  implies  that  the  BSD  Seminar  is  providing 
meaningful  guidance  to  attendees  in  differing  areas. 

4.  Eased  upon  the  consensus  of  the  comments  from 
past  attendees,  the  BSD  Seminar  is  rated  highly  as  a 
course  which  provides  valuable  guidance  for  the  new 


supervisor. 
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Phase  II 

Conclusions  thaf  can  be  drawn  from  the  results  of 
Phase  II  are  much  more  tenuous. 

1  .  The  external  validity  of  Phase  II  lacks  strong 
support  in  several  areas.  The  fact  that  data  from  only 
two  classes  could  be  used,  and  that  the  sample  sizes  of 
those  returning  post-tests  were  less  than  optimum, 
severely  limits  the  generalization  of  the  results. 

2.  The  original  null  hypothesis  was  rejected.  For 
the  two  classes  involved,  the  BSD  Seminar  did  have  a 
significant  effect  on  the  attendees'  leadership  styles  as 
measured  by  the  T-P  Leadership  Questionnaire.  The  ten¬ 
dency  in  both  classes  was  to  increase  the  scores  in  their 
concern  for  people.  This  is  consistent  with  past  research 
that  indicates  this  trend  often  occurs  with  human 
relations-oriented  leadership  training.  There  was  also 
consistency  between  classes  in  the  change  of  task  scores, 
but  it  was  in  the  opposite  direction.  Although  task 
scores  decreased  ,  because  they  were  consistently  higher 
than  people  scores  on  the  pre-tests ,  the  end  result  was 

to  more  c]osely  balance  the  two  scores  on  the  post-tests. 
This  equal  concern  for  people  and  task  is  characteristic 
of  participative  leadership. 

3.  While  it  appears  from  the  conclusion  above  that 


i*8 

Dr.  Solomon's  goal  towards  participative  leadership  is 
being  achieved,  before  generalizing  these  results  several 
factors  must  be  considered.  Pretest-treatment  interaction 
could  also  have  contributed  to  the  results.  The  simple 
fact  that  the  pre-test  was  given  may  have  increased  the 
attendees'  awareness  to  their  people-task  orientation. 
Similarly,  the  Hawthorne  effect,  the  fact  that  they  were 
involved  in  a  special  study,  could  have  influenced  the 
results  also. 

4.  Based  solely  on  the  resu.lts  from  the  scores  of 
the  two  classes  involved,  there  does  not  appear  to  be  a 
significant  difference  in  the  effects  cf  the  fou’-'-day 
format  seminar  versus  the  five-day  format. 

Recommendations 

Within  the  broad  guidelines  established  by  the 
Naval  Civilian  Personnel  Command,  all  indications  from 
this  program  evaluation  imply  that  the  BSD  Seminar 
is  meeting  its  objectives  effectively.  It  is  felt  that 
some  of  the  information  obtained  during  this  study  can 
be  useful  in  making  decisions  on  the  direction  this 
seminar  may  take  in  the  future.  However,  several  recom¬ 
mendations  must  be  made  with  regards  to  the  results  of 
this  evaluation  and  to  others  that  might  follow. 
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Valid  evaluation  of  the  effectiveness  of  any  course 
which  lacks  clearly  stated,  measurable  objectives  is 
extremely  difficult.  Unfortunately,  many  courses  today 
in  civilian  and  military  organizations  alike,  are 
established  with  this  inherent  weakness.  To  prevent 
the  valuable  loss  of  both  time  and  money  due  to  ineffec¬ 
tive  or  misdirected  courses  ,  all  programs  should  establish 
clear  instructional  objectives  that  can  be  used  as 
accurate  yardsticks  to  measure  their  success. 

The  theoretical  nature  of  leadership  training  lends 
to  the  on-going  controversy  over  the  most  effective 
styles  and  techniques  for  its  measurement.  While  the 
T-P  Leadership  Questionnaire  utilized  in  this  study 
may  accurately  measure  an  individual's  concern  for  people 
and  task,  it  must  be  realized  that  these  represent 
attitudes  and  not  necessarily  behaviors.  To  more  accu¬ 
rately  measure  the  results  and  effects  of  this  seminar 
on  their  leadership  styles,  the  attendees  would  have  to 
be  observed  in  actual  leadership  situations  before  and 
after  the  seminar.  Unfortunately,  such  an  involved 
process  was  beyond  the  scope  of  this  evaluation. 

Nonetheless,  valuable  inferences  could  be  made  from 
the  type  of  resuits  obtained  in  Phase  II  of  this  study. 
However,  more  replications  would  be  required  before  the 
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validity  of  generalizing  those  results  could  be  strongly 
supported . 

Several  specific  recommendations  can  be  made  if 
replications  are  attempted  in  the  future.  Judging  from 
the  differences  in  the  percentages  of  questionnaires 
returned  between  Phases  I  and  II,  it  appears  that  mail¬ 
ing  to  the  work  place  rather  than  the  home  reaps 
sxg.nif ica.nt ly  better  results. 

While  more  replications  of  Phase  II  might  yield  a 
significant  difference  in  scores  between  the  four-day 
and  five-day  formats  of  the  seminar,  this  alone  would  not 
indicate  superiority  of  one  format  over  the  other.  The 
material  presented  during  the  additional  day  covers  much 
more  than  just  participative  leadership.  Therefore, 
because  of  this  varied  content,  additional  evaluation 
techniques  would  be  required  to  determine  if  there  is 
justification  for  the  extra  day  of  instruction. 
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BASIC  SUPERVISORY  DEVELOPMENT 
QUESTIONNAIRE 


Lisieo  below  ore  six  topics  that  were  covered  in  Dr.  Solomon's  "5asic 
wper visorv  Development"  seminar.  Based  upon  your  job  experiences  since  taking 
he  coursei  indicate  with  an  the  extent  to  which  you  -feel  these  areas  were 
adeauateiv  cove-ec:  Requires  More  Emphasis  (More),  Adequately  Covered  (Adeq), 
Requires  Less  Emphasis  (Less). 


1.  Historical  background / e :: posure  to  various 
theories  of  management  (Scientific  Management, 
Maslow's  Hierarchy  of  Needs,  Theory  X-Theory 
Y,  etc.). 


2.  The  Supervisor  as  Facilitator  (Stating  the 
problem, Supplying  essential  facts,  Defining  the 
area  of  freedom,  etc.) 


3.  Functions  of  Supervision  (Planning, 
Organizing,  Controlling). 

4.  Decision-making  ana  Problem-solving 
(Autocratic,  Consultative,  and  Group 
Approaches). 

5.  Supervision  through  Participation 
(Permissiveness  and  Controls). 


fr.  Interviewing  and  Counseling  (Objectives, 
Pitfalls,  Active  Listening,  etc.). 


0*  tne  si:;  tooics  listed  above,  choose  the  one  which  you  feel  hid  the 
greatest  impact  on  your  leadership  style /supervisory  effectiveness  as  a  result  of 
taking  the  seminar.  Please  circle  the  number  of  this  topic. 


A's  vou  currently  working  m  a  supervisory  position'''  Yes _  Nc 


Female 
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Ntme 


Dete _ 


Dirtc  tions:  Tht  following  items  describe  septets  of  ltsdtrthlp  bthtvior. 

Respond  to  etch  item  4Ccording  to  the  wsy  you  would  most  liKtly  4ct  if  you 
w*p«  tht  lt4dtr  of  4  worK  group.  Circle  whtthtr  you  would  most  liKtly 
behtvt  in  tht  dttcribtd  wty:  slwtys  <A)»  frtqutntly  (F)»  occtsionslly  <0)» 
seldom  <S)i  or  ntvtr  (N). 

A  F  0  S  N  i.  I  would  most  likely  set  4t  the  spokesmen  of  the  group. 


A  F  O  S  N 
A  F  0  S  N 
A  F  0  S  N 
A  F  0  S  N 


2.  I  would  encoursgt  ovtrtimt  work. 

3.  I  would  tllow  mtmbtrt  complttt  fretdom  in  thtir  work. 

4.  I  would  encoursgt  tht  utt  of  uniform  proctdurts. 

5.  I  would  permit  tht  mtmbtrt  to  utt  thtir  own  judgtmtnt  in 


A  F  O  S  N 
A  F  O  S  N 
A  F  0  S  N 
A  F  0  S  N 
A  F  0  S  N 


solving  probltmt. 

6.  I  would  strstt  bting  4ht4d  of  competing  groups. 

7.  I  would  spesk  45  4  rtprtstnt4tivt  of  tht  group. 

3.  I  would  nttdlt  mtmbtrt  for  grester  tffort. 

9.  I  would  try  out  my  idt4t  in  tht  group. 

10.  I  would  ltt  tht  mtmbtrt  do  thtir  work  the  way  they  think 


A  F  0  S  N 
A  F  0  S  N 
A  F  0  S  N 
A  F  0  S  N 
A  F  0  S  N 

A  F  0  S  N 
A  F  0  S  N 


best. 

11.  I  would  bt  working  herd  for  4  promotion. 

12.  I  would  toltr4tt  postpontment  4nd  uncert4inty. 

13.  I  would  spt4k  for  tht  group  if  thtrt  wtrt  visitors  prtttnt. 

14.  I  would  kttp  tht  work  moving  4t  4  rspid  psce. 

13.  I  would  turn  the  mtmbtrt  loott  on  4  job  4nd  ltt  them  go  to 
it. 

16.  I  would  settle  conflicts  whtn  thty  occur  in  tht  group. 

17.  I  would  get  swtmped  by  detsils. 

18.  I  would  represent  the  group  4t  outside  meetings. 


A  F  0  S  N 


A  F  0  S  N 
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19,  I  would  bt  reluct Ant  to  Allow  tht  members  Any  freedom  of 
Action. 

A  F  0  S  N  20.  I  would  decide  whAt  should  bt  dent  And  how  it  should  bt 
dont. 

A  F  0  S  N  21.  1  would  push -for.  inert AStd  production. 

A  F  0  S  N  22.  I  would  ltt  some  members  hAvt  Authority  which  I  could  kttp. 

A  F  0  S  N  23.  Things  would  ususlly  turn  out  as  1  hAd  prtdicttd. 

A  F  0  S  N  24.  I  would  Allow  tht  group  a  high  dtgrtt  of  initistivt. 

A  F  0  S  N  25.  I  would  Assign  group  members  to  pArticuisr  tssks. 

A  F  0  S  N  26.  I  would  bt  willing  to  mskt  chAngts. 

A  F  0  S  N  27.  I  would  Ask  tht  mtmbtrs  to  work  hArdtr. 

A  F  0  S  N  28.  I  would  trust  the  group  mtmbtrs  to  exercise  good  judgmtnt. 

A  F  0  S  N  29.  I  would  schedule  the  work  to  bt  done. 

A  F  0  S  N  30.  I  would  refuse  to  explAin  my  Actions. 

A  F  0  S  N  31.1  would  ptrsuAde  others  thst  my  idess  Art  to  their 
AdvsntAge. 

A  F  0  S  N  32.  I  would  permit  the  group  to  set  its  own  pact. 

A  F  0  S  N  33.  I  would  urge  the  group  to  best  its  previous  record. 

A  F  0  S  N  34.  I  would  Act  without  consulting  the  group. 

35.  I  would  Ask  thAt  group  members  follow  stAndsrd  rules  And 
regulAtions. 


A  F  O  S  N 
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DtPARTTv’iENl  OF  1  HL  !;AVY 
\  ,iL  :»v  :uan  Fcr<bou.'j:_  command 
SOUTHWEST  REGION 
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jMN  CIEGC.  CAUPORN.i.  ?2ift  1  N  «;  E  w  v  f  - !  *-  r  K  T 


NCPCSWR : RAE : ab 
12410 

11  March  1985 


From:  Training  Director,  Naval  Civilian  Personnel  Command, 

Southwest  Region 

To:  Participants 

Sub  j :  BASIC  SUPERVISORY  DEVELOPMENT 

Enel:  (1)  Questionnaire 

1.  Within  the  past  year  you  attended  a  seminar  on  "Basic 
Supervisory  Development "  given  by  Dr.  Lawrence  N.  Solomon.  The 
enclosed,  brief  questionnaire  is  part  of  a  current  projected 
aimed  at  measuring  the  relevance  and  effectiveness  of  this 
course  in  its  goal  to  improve  supervisory  skills.  Your 
accurate.  honest  responses  are  imperative  to  the  success  of 
this  project  and  to  the  future  improvement  of  the  seminar. 
Please' take  the  few  minutes  required  to  answer  the  questionnaire 
new,  and  return  it  in  the  enclosed  pre-addressed  envelope. 
Thank  you  for  your  time  and  assistance. 


A.  EVANS 
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PRE-TEST  INSTRUCTIONS 


6H 


In  a  moment  I  will  be  passing  out  a  brief  questionnaire  that  is  designed  to 
measure  certain  aspects  of  leadership  behavior.  This  is  part  of  a  current  project 
aimed  at  measuring  the  effects  of  this  course  on  your  individual  leadership  style. 
As  a  follow-up  to  this  initial  measurement,  in  approximately  one  month  you  will 
receive  a  similar  questionnaire  in  the  mail.  We  are  requesting  that  you  take  the 
few  moments  reauired  to  complete  this  form  and  then  return  it  in  the  enclosed 
pre-addressed  envelope.  The  success  of  the  entire  project  depends  upon  your 
honest  answers.  Since  you  will  be  taking  the  follow-up  questionnaire  without 
supervision,  it  is  requested  that  you  attempt  to  do  it  promptly  and  under  similar 
conditions  to  this  initial  situation.  Try  to  take  it  at  approximately  the  same  time 
in  the  morning,  without  interruptions,  and  spend  about  the  same  amount  of  time 
or.  the  entire  process.  It  is  important  that  you  place  your  name  and  the  date  at 
the  top  of  the  page. 

Once  again  it  is  stressed  that  the  purpose  of  this  project  is  strictly  for 
evaluative  research  with  future  course  improvement  as  its  goal.  Individual 
results  will  be  maintained  in  the  strictest  confidentiality  by  the  researcher. 
Thank  you  for  your  time  and  assistance. 
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DEPARTMENT  OF  THE  NAVY 

NAVAL  CIVILIAN  PERSONNEL  COMMAND 
SOUTHWEST  REGION 
MO  FRONT  STREET 
SUITE  5-S-I* 

SAN  OIEGO,  CALIFORNIA  ItlU  IN  REPLY  REFER  TO 

NCPCSWR :  DEJ :  ym 
12410 

29  May  1985 


From:  Training  Director,  Naval  Civilian  Personnel  Command, 

Southwest  Region 
To:  Participants 

Subj:  BASIC  SUPERVISE  DEVELOPMENT 

Enel:  (1)  Questionnaire 

1.  Approximately  four  weeks  ago  you  attended  NCPC's  Basic  Supervisory 
Development  Seminar.  At  that  time  you  were  given  a  brief  questionnaire 
as  part  of  a  current  project  aimed  at  measuring  the  effects  of  this 
course  on  your  individual  leadership  style.  The  second  half  of  this 
project  requires  that  you  take  3  similar  questionnaire  at  this  time.  We 
are  requesting  that  you  take  the  few  moments  required  to  complete  this 
form  and  then  return  it  in  the  enclosed  pre-addressed  envelope.  The 
success  of  the  entire  project  depends  upon  your  honest  answers,  not 
what  you  think  is  expected.  It  is  not  important  to  remember  how  you 
answered  the  initial  questionnaire. 

2.  Since  you  will  be  taking  the  follow-up  questionnaire  without 
su^rvision,  it  is  requested  that  you  attempt  to  do  it  promptly  and 
under  similar  conditions  to  the  initial  situation.  Try  to  take  it  at 
approximately  the  same  time  in  the  morning,  without  interruptions,  and 
spend  about  the  same  amount  of  time  on  the  entire  process.  It  is 
important  that  you  place  your  name  (as  you  did  on  the  initial 
questionnaire)  and  the  date  at  the  top  of  the  page.  This  is  strictly 
for  statistical  purposes  so  that  we  can  match  the  initial  with  the 
follow-up  questionnaires. 

3.  Once  again  it  is  stressed  that  the  purpose  of  this  project  is 
strictly  for  evaluative  research  with  future  course  improvement  as  its 
goal.  Individual  results  will  be  maintained  in  strict  confidentiality  by 
the  researcher.  Thank  you  for  your  time  and  cooperation. 
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APPENDIX  F 

LEADERSHIP  QUESTIONNAIRE  SCORING  PROCEDURES 


(Excerpted  from  Pfeiffer  &  Cones,  A  Handbook  of 
Structured  Experiences  for  Human  Relations  Training, 

1  97 A  .  ) 

IV.  The  facilitator  instructs  the  participants 

in  the  scoring  as  follows: 

1.  Circle  the  item  number  for  items  8, 

12,  17,  18,  19,  30,  3ij,  and  35. 

2.  Write  the  number  1  in  front  of  a 
circled  item  number  if  you  responded 
S  (seldom )  or  N  (never )  to  that  item. 

3.  Also  write  a  number  1  in  front  of 
item  numbers  not  circled  if  you 
responded  A  (always!  or  F  (frequently). 

A.  Circle  the  number  1's  which  you  have 
written  in  front  of  the  following 
items  3,  5,  8,  10,  15,  18,  19,  22, 

24,  26,  28,  30,  32,  3A,  and  35. 

5  .  Count  the  circled  number  1's.  This 
is  your  score  for  concern  for  people. 
Record  the  score  in  the  blank 
following  the  letter  P  at  the  end 
of  the  questionnaire. 

6 .  Count  the  uncircled  number  1's. 

This  is  your  score  for  concern  for 
task.  Record  this  number  in  the 
blank  following  the  letter  T. 


